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Administrivia 

• PSl is out! On the web site... 

• And... because I was not here last Thurs and so we’re a 
lecture “behind” the due date is pushed back until Tues 
evening Sept 9 th , 11:55pm. But don’t wait until Sunday 
please... 
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Last time: Convolution 


Convolution: 

• Flip the filter in both dimensions (right to left, bottom to top) 

• Then apply cross-correlation 
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Notation for 
convolution 
operator 


K. Grauman 
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Convolution vs. correlation 


Convolution 

k k 

G[i,j]= £ £ H[u,v]F[i-u,j-v] 

u— — k v— — k 

G = H*F 

(Cross-)correlation , r 

k k T 

G[iJ]= £ £ H[u,v]F[i + u,j + v] 

u— — k v— — k 

G = H®F 

• When H is symmetric, no difference. We tend to use the terms 
interchangeably. 

• Convolution with an impulse (centered at 0,0) is the identity 


K. Grauman 
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Filters for features 

• Previously, thinking of filtering 
as a way to remove or reduce 

noise 

• Now, consider how filters will 
allow us to abstract higher- 
level “features”. 

• Map raw pixels to an intermediate 
representation that will be used for 
subsequent processing 

• Goal: reduce amount of data, 
discard redundancy, preserve 
what’s useful 




K. Grauman 
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Template matching 

• Filters as templates: 

Note that filters look like the effects they are intended to 
find — “matched filters” 



• Use ( normalized) cross-correlation score to find a given 
pattern (template) in the image. 

• Normalization needed to control for relative brightness. More in 
problem sets. 


K. Grauman 
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Template matching 



Template (mask) 

Scene 

A toy example 


K. Grauman 
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Template matching 



Detected template 
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Template 


K. Grauman 
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Template matching 



Detected template 
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Correlation map 


K. Grauman 
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Where’s Waldo? 



Scene 



Template 


K. Grauman 
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Where’s Waldo? 



Detected template 



Template 


K. Grauman 
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Template demo... 

• In directory C:\Bobick\matlab\CS4495\Filter 

• echodemo waldotemplate 
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Correlation map 


K. Grauman 
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Template matching 



Scene 



Template 


What if the template is not identical to some 
subimage in the scene? 


K. Grauman 
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Template matching 




Template 


Detected template 

Match can be meaningful, if scale, orientation, 
and general appearance is right. 


K. Grauman 
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Generic features... 

• When looking for a specific object or pattern, the features 
can be defined for that pattern - we will do this later in the 
course for specific object recognition. 

• But for generic images, what would be good features? 
What are the parts or properties of the image that encode 
its “meaning” for human (or other biological) observers? 

• Some examples of greatly reduced images... 
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Edges seem to be important... 
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Origin of Edges 



surface normal discontinuity 
depth discontinuity 
surface color discontinuity 
illumination discontinuity 


• Edges are caused by a variety of factors 

• Information theory view: edges encode change, change 
is what is hard to predict, therefore edges efficiently 
encode an image 
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In a real image 



Reflectance change: 
appearance 
information, texture 


Change in surface 
orientation: shape 


ast shadows 


Depth discontinuity 
object boundary 
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• Convert a 2D image into a set of curves 

• Extracts salient features of the scene 

• More compact than pixels 
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Edge detection 



• How can you tell that a pixel is on an edge? 




CS 4495 Computer Vision - A. Bobick 


Templates/Edges 


Images as functions. . . 



Edges look like steep cliffs 
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Edge Detection 


Basic idea: look for a neighborhood with 
strong signs of change. 


Problems: 

• neighborhood size 
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how to detect change 
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Derivatives and edges 

An edge is a place of rapid change in the image 
intensity function. 


image 


intensity function 

(along horizontal scanline) first derivative 



edges correspond to 
extrema of derivative 


Source: L. Lazebnik 
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Differential Operators 

• Differential operators - here we mean some operation that when 
applied to the image returns some derivatives. 

• We will model these “operators” as masks/kemels which when 
applied to the image yields a new function that is the image 

gradient function. 


• We will then threshold the this gradient function to select the 
edge pixels. 

• Which brings us to the question: 

What’s a gradient? 
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Image gradient 

The gradient of an imaae: 


v/ 


df df 

_dx 5 dy _ 


v/ 
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dx’ U 


V/ 


0 ^ 

u ’ 9y 
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The gradient points in the direction of most rapid increase in 


The gradient direction is given by: 


e 


tan -1 

tan \dy/dx 


• how does this relate to the direction of the edge? 


The edge strength is given by the gradient magnitude 



= 'df df 

_dx' > dy . 


intensity 
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Discrete gradient 

• For 2D function, f(x,y), the partial derivative is: 

gfjXy) = j im f(x + £,y)~ f(x,y) 

dx s ^° £ 

• For discrete data, we can approximate using finite 
differences: 

df(*,y) ,, f(x + ly)~ f(x,y ) 

1 

~ f y) — f (x, y) “right derivative” 

But is it??? 


dx 
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Finite differences 



Computer Vision - A Modern Approach 
Set: Linear Filters 
Slides by D.A. Forsyth 


Templates/Edges 



CS 4495 Computer Vision - A. Bobick 


Templates/Edges 


Partial derivatives of an image 



df(x,y) 



Which shows changes with respect to x? 


(showing correlation filters) 
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Differentiation and convolution 

• For 2D function, f(x,y), the partial derivative is: 

gfjXy) = j im f(x + £,y)~ f(x,y) 

dx s ^° s 

• For discrete data, we can approximate using finite 
differences: 

df(*,y) rj f(x + l,y)~ f(x,y ) 

dx 1 


• To implement above as convolution, what would be the 
associated filter? 
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The discrete gradient 

• We want an “operator” (mask/kernel) that we can apply to 
the image that implements: 

gfjXy) = lim f(x + £,y)~ f(x,y ) 

dx £ ^° s 


How would you implement this as a cross-correlation? 


(not flipped) 


a 



middle” pixel? 


0 

0 

Not symmetric 
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0 

point; which is 
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0 



Average of 
“left” and 
“right” 
derivative . 
See? 
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Example: Sobel operator 


1 

8 


On a pixel of the image I 

•Let g x be the response to mask S x (sometimes * 1/8) 
•Let g be the response to mask S 
What is the gradient? 

(Sobel) Gradient is VI = [g x g y ] T 

g = (g x 2 + g 2 ) 1/2 is the gradient magnitude. 

0 = atan2(g , g x ) is the gradient direction. 
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Sobel Operator on Blocks Image 





blocks.jpg.Sobel.Threshold 


nr B 


blocks.jpg.Sobel 


blocks.jpg 




original image 


gradient 

magnitude 


thresholded 

gradient 

magnitude 
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Some Well-Known Masks for Computing 
Gradients 

Sx Sy 


• Sobel: 


Prewitt: 
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Matlab does edges 


» My = f special ( 'sobel' ) ; 

» outim = imfilter( double (im) , My); 
» imagesc (outim) ; 

» colormap gray; 
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But... 

• Consider a single row or column of the image 

• Plotting intensity as a function of x 



• Apply derivative operator.... 



Uh, where’ 
the edge? 


D 
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800 1000 1200 1400 1600 1800 2000 
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Finite differences responding to noise 





Increasing noise -> 

(this is zero mean additive gaussian noise) 


D. Forsyth 






CS 4495 Computer Vision - A. Bobick 


Templates/Edges 


Solution: smooth first 






Kernel 
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Derivative theorem of convolution 


d 


d 


This saves us one operation: * f)~ * f 


Sigma = 50 



0 200 400 600 800 1000 1200 1400 1600 1800 2000 


h 






How can we find (local) maxima of a function? 
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2 nd derivative of Gaussian 


Consider 


d 


2 


dx 
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(h* f ) 
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Sigma = 50 
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Where is the edge? Zero-crossings of bottom graph 
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What about 2D? 
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Derivative of Gaussian filter - 2D 
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Derivative of Gaussian filter - 2D 
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Why is this preferable? 



2 
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Derivative of Gaussian filters 



x-direction 



Is this for correlation 
or convolution ? 



y-direction 



And for y it’s always 
a problem! 

Source: L. Lazebnik 
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Smoothing with a Gaussian 


Parameter a is the “scale” / “width” / “spread” of the Gaussian 
kernel, and controls the amount of smoothing. 



for sigma=l:3:10 

h = fspecial( ' gaussian' , fsize, sigma); 
out = imfilter(im, h); 
imshow(out ) ; 
pause; 
end 
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Smoothing with a Gaussian 

Recall: parameter a is the “scale” / “width” / “spread” of the 
Gaussian kernel, and controls the amount of smoothing. 





0 10 20 30 


0 10 20 30 


0 10 20 30 
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Effect of a on derivatives 



a = 1 pixel a = 3 pixels 


The apparent structures differ depending on 
Gaussian’s scale parameter. 


Larger values: larger scale edges detected 
Smaller values: finer features detected 





CS 4495 Computer Vision - A. Bobick 


Templates/Edges 


Gradients -> edges 

• Primary edge detection steps: 

• 1. Smoothing: suppress noise 

• 2. Edge “enhancement”: filter for contrast 

• 3. Edge localization 

• Determine which local maxima from filter output are actually edges 
vs. noise 

• Threshold, Thin 
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Canny edge detector 

Filter image with derivative of Gaussian 
Find magnitude and orientation of gradient 

Non-maximum suppression: 

• Thin multi-pixel wide “ridges” down to single pixel 
width 

Linking and thresholding (hysteresis): 

• Define two thresholds: low and high 

• Use the high threshold to start edge curves and the 
low threshold to continue them 

MATLAB: edge (image, 'canny'); 

»help edge 


Source: D. Lowe, L. Fei-Fei 
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The Canny edge detector 



original image (Lena) 
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The Canny edge detector 



magnitude of the gradient 
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The Canny edge detector 
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thresholding 
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The Canny edge detector 


Problem: 
pixels along 
this edge 
didn’t 

survive the 
thresholding 



thinning 

(non-maximum suppression) 
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The Hannw orlnp Hotart 





How to turn 
these thick 
regions of 
the gradient 



curves? 
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Non-maximum suppression 



• Check if pixel is local maximum along gradient 
direction 

• can require checking interpolated pixels p and r 
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The Canny edge detector 



thinning 

(non-maximum suppression) 
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Effect of a (Gaussian kernel spread/size) 



original Canny with a = 1 Canny with <7 = 2 


The choice of a depends on desired behavior 

• large a detects large scale edges 

• small a detects fine features 
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So , what scale to choose? 


It depends what we’re looking for. 



Too fine of a scale... can’t see the forest for the trees. 

Too coarse of a scale... can’t tell the maple grain from the cherry. 
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Single 2D edge detection filter 




Laplacian of Gaussian 


Gaussian 


derivative of Gaussian 


h a (u , v) 
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2 : 


V is the Laplacian operator. Edges at zero crossings 
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Edge demo in CS4495/Edges 
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End CS4495 



